Method and apparatus for detection using gradient-weighted and/or distance-weighted graph cuts

ABSTRACT

Methods and apparatuses process images. The method according to one embodiment accesses digital image data representing an image including an object; generates a connected graph associated with the image, the generating step including representing pixels of the image in a higher than two dimensional space to obtain pixel representations, generating a pixel representation graph using the pixel representations, and assigning weights to edges between the pixel representations in the pixel representation graph, based on a gradient characteristic between the pixel representations, to obtain a connected graph; and segments the connected graph using an energy minimizing function, to obtain pixels of the image associated with the object.

CROSS REFERENCE TO RELATED APPLICATION

This non-provisional application is related to co-pendingnon-provisional application titled “Method and Apparatus for DetectionUsing Cluster-Modified Graph Cuts” filed concurrently herewith, theentire contents of which are hereby incorporated by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a digital image processing technique,and more particularly to a method and apparatus for processing andsegmenting areas in images.

2. Description of the Related Art

Mammography images are powerful tools used in the diagnosis of medicalproblems of breasts. An important feature in mammography images is thepectoral muscle area. The pectoral muscle area can be used to identifybreast abnormalities, can facilitate comparative analysis and processingof mammography images, can convey significant information relating tobreast position and deformation, etc.

For example, the pectoral muscle in the medio-lateral view, as istypically shown in mammograms, is a major landmark for mammogramreconstruction and registration. The pectoral muscle also helps quantifyquality of mammograms and can be used for automatic quality assurance.The area overlying the pectoral muscle is a highly suspicious zone fordevelopment of masses and is checked by radiologists for false-negatives(non-cancerous areas). Also, mammographic parenchymal patterns, whichare a marker of breast cancer risk, and the pectoral muscle haveidentical texture, which leads to false-positives in detection ofmalignant masses.

Pectoral muscle identification is a non-trivial task due to variabilityof borders and contrast, and unclear areas in breast images.Typical/conventional methods to detect the pectoral muscle rely on someheuristic gradient measures to fit a line to the pectoral muscleboundary. One such pectoral muscle detection technique is described inthe publication “Automatic Pectoral Muscle Segmentation on MediolateralOblique View Mammograms”, by S. Kwok, R. Chandrasekhar, Y. Attikiouzeland M. Rickard, IEEE Transactions on Medical Imaging v.23 No. 9,September 2004. In the technique described in this publication, anadaptive algorithm is proposed to automatically extract the pectoralmuscle in digitized mammograms. The adaptive algorithm uses knowledgeabout the position and shape of the pectoral muscle on medio-lateraloblique views. The pectoral edge is first estimated by a straight linethat is validated for correctness of location and orientation. Thelinear estimate is then refined using iterations. This technique,however, relies on detailed pre-operational knowledge of the digitaldata, to tune algorithms for the data at hand, and requires adjustmentsof many parameters. This technique force-fits a line or curve to thepectoral muscle boundary, even though the boundary may not be amenableto curve fitting.

Disclosed embodiments of this application address these and other issuesby using a method and an apparatus for pectoral muscle detection usinggradient-weighted and/or distance weighted graph cuts. The method andapparatus segment graphs associated with breast images, to obtain pixelsassociated with the pectoral muscle. The method and apparatus areapplicable to breast images with various views, and do not requiretuning beforehand. The method and apparatus are applicable to othertypes of images besides breast images, and detect various objectsincluded in images.

SUMMARY OF THE INVENTION

The present invention is directed to methods and apparatuses forprocessing images. According to a first aspect of the present invention,an image processing method comprises: accessing digital image datarepresenting an image including an object; generating a connected graphassociated with the image, the generating step including representingpixels of the image in a higher than two dimensional space to obtainpixel representations, generating a pixel representation graph using thepixel representations, and assigning weights to edges between the pixelrepresentations in the pixel representation graph, based on a gradientcharacteristic between the pixel representations, to obtain a connectedgraph; and segmenting the connected graph using an energy minimizingfunction, to obtain pixels of the image associated with the object.

According to a second aspect of the present invention, an imageprocessing method comprises: accessing digital image data representingan image including an object; generating a connected graph associatedwith the image, the generating step including representing pixels of theimage to obtain pixel representations, generating a pixel representationgraph using the pixel representations, and assigning weights to edgesbetween the pixel representations in the pixel representation graph,based on a gradient characteristic between the pixel representations, toobtain a connected graph; and segmenting the connected graph using anenergy minimizing function, to obtain pixels of the image associatedwith the object.

According to a third aspect of the present invention, an apparatus forprocessing images comprises: an image data input unit for accessingdigital image data representing an image including an object; a graphgeneration unit for generating a connected graph associated with theimage, the graph generation unit generating a connected graph byrepresenting pixels of the image to obtain pixel representations,generating a pixel representation graph using the pixel representations,and assigning weights to edges between the pixel representations in thepixel representation graph, based on a gradient characteristic betweenthe pixel representations, to obtain a connected graph; and asegmentation unit for segmenting the connected graph using a max-flowsegmentation to obtain pixels of the image associated with the object.

BRIEF DESCRIPTION OF THE DRAWINGS

Further aspects and advantages of the present invention will becomeapparent upon reading the following detailed description in conjunctionwith the accompanying drawings, in which:

FIG. 1 is a general block diagram of a system including an imageprocessing unit for detection using weighted graph cuts according to anembodiment of the present invention;

FIG. 2 is a block diagram illustrating in more detail aspects of theimage processing unit for detection using weighted graph cuts accordingto an embodiment of the present invention;

FIG. 3 is a flow diagram illustrating operations performed by an imageprocessing unit for detection using weighted graph cuts according to anembodiment of the present invention illustrated in FIG. 2;

FIG. 4 is a flow diagram illustrating exemplary image processingoperations performed by an image operations unit and a cropping unitincluded in an image processing unit for detection using weighted graphcuts according to an embodiment of the present invention illustrated inFIG. 2;

FIG. 5A illustrates an exemplary mammogram image;

FIG. 5B illustrates exemplary mammogram images obtained after imageprocessing operations and cropping according to an embodiment of thepresent invention illustrated in FIG. 4;

FIG. 6 is a flow diagram illustrating operations performed by a graphgeneration unit included in an image processing unit for detection usingweighted graph cuts according to an embodiment of the present inventionillustrated in FIG. 2;

FIG. 7A is a flow diagram illustrating exemplary operations forrepresentation of pixels for a region of interest by a graph generationunit included in an image processing unit for detection using weightedgraph cuts according to an embodiment of the present inventionillustrated in FIG. 6;

FIG. 7B is a flow diagram illustrating exemplary operations forselection of a source point and a sink point in a region of interest bya graph generation unit included in an image processing unit fordetection using weighted graph cuts according to an embodiment of thepresent invention illustrated in FIG. 6;

FIG. 7C is a flow diagram illustrating exemplary operations for creatinga graph with weighted edges for a region of interest by a graphgeneration unit included in an image processing unit for detection usingweighted graph cuts according to an embodiment of the present inventionillustrated in FIG. 6;

FIG. 8 is a flow diagram illustrating operations performed by asegmentation unit included in an image processing unit for detectionusing weighted graph cuts according to an embodiment of the presentinvention illustrated in FIG. 2; and

FIG. 9 is a flow diagram illustrating operations performed by a imagerestoration unit included in an image processing unit for detectionusing weighted graph cuts according to an embodiment of the presentinvention illustrated in FIG. 2.

DETAILED DESCRIPTION

Aspects of the invention are more specifically set forth in theaccompanying description with reference to the appended figures. FIG. 1is a general block diagram of a system including an image processingunit for detection using weighted graph cuts according to an embodimentof the present invention. The system 100 illustrated in FIG. 1 includesthe following components: an image input unit 20; an image processingunit 30; a display 60; an image output unit 50; a user input unit 70;and a printing unit 40. Operation of the system 100 in FIG. 1 willbecome apparent from the following discussion.

The image input unit 20 provides digital image data. The digital imagedata may be a medical image, a mammography image, an image of people,etc. Image input unit 20 may be one or more of any number of devicesproviding digital image data derived from a radiological film, adiagnostic image, a digital system, etc. Such an input device may be,for example, a scanner for scanning images recorded on a film; a digitalcamera; a digital mammography machine; a recording medium such as aCD-R, a floppy disk, a USB drive, etc.; a database system which storesimages; a network connection; an image processing system that outputsdigital data, such as a computer application that processes images; etc.

The image processing unit 30 receives digital image data from the imageinput unit 20 and performs object detection using weighted graph cuts ina manner discussed in detail below. A user, e.g., a radiology specialistat a medical facility, may view the output of image processing unit 30,via display 60 and may input commands to the image processing unit 30via the user input unit 70. In the embodiment illustrated in FIG. 1, theuser input unit 70 includes a keyboard 75 and a mouse 77, but otherconventional input devices could also be used.

In addition to performing detection using weighted graph cuts inaccordance with embodiments of the present invention, the imageprocessing unit 30 may perform additional image processing functions inaccordance with commands received from the user input unit 70. Theprinting unit 40 receives the output of the image processing unit 30 andgenerates a hard copy of the processed image data. In addition or as analternative to generating a hard copy of the output of the imageprocessing unit 30, the processed image data may be returned as an imagefile, e.g., via a portable recording medium or via a network (notshown). The output of image processing unit 30 may also be sent to imageoutput unit 50 that performs further operations on image data forvarious purposes. The image output unit 50 may be a module that performsfurther processing of the image data, a database that collects andcompares images, etc.

FIG. 2 is a block diagram illustrating in more detail aspects of theimage processing unit 30 for detection using weighted graph cutsaccording to an embodiment of the present invention. As shown in FIG. 2,the image processing unit 30 according to this embodiment includes: animage operations unit 115; a cropping unit 125; a graph generation unit135; a segmentation unit 145; and an image restoration unit 155.Although the various components of FIG. 2 are illustrated as discreteelements, such an illustration is for ease of explanation and it shouldbe recognized that certain operations of the various components may beperformed by the same physical device, e.g., by one or moremicroprocessors.

Generally, the arrangement of elements for the image processing unit 30illustrated in FIG. 2 performs preprocessing and preparation of digitalimage data, generation of graphs for the digital image data, anddetection of various areas in the digital image data using the generatedgraphs. Operation of image processing unit 30 will be next described inthe context of mammography images, for detection of pectoral muscleareas. However, the principles of the current invention apply equally toother areas of image processing, and to detection of other types ofobjects in digital image data.

Image operations unit 115 receives a mammography image from image inputunit 20 and may perform preprocessing and preparation operations on themammography image. Preprocessing and preparation operations performed byimage operations unit 115 may include resizing, cropping, compression,color correction, etc., that change size and/or appearance of themammography image.

Image operations unit 115 sends the preprocessed mammography image tocropping unit 125, which crops a part of the mammography image. Graphgeneration unit 135 receives a cropped section of the mammography imageand generates a graph associated with the cropped section, after whichsegmentation unit 145 segments the graph to obtain an area of thepectoral muscle in the mammography image. Finally, image restorationunit 155 outputs a breast image with identified pectoral muscle pixels.Image restoration unit 155 may also send results of pectoral areasegmentation back to cropping unit 125 or to image operations unit 115,for refinement of results of pectoral muscle detection.

The output of image restoration unit 155 may be sent to image outputunit 50, printing unit 40, and/or display 60. Operation of thecomponents included in the image processing unit 30 illustrated in FIG.2 will be next described with reference to FIGS. 3-9.

Image operations unit 115, cropping unit 125, graph generation unit 135,segmentation unit 145, and image restoration unit 155 are softwaresystems/applications. Image operations unit 115, cropping unit 125,graph generation unit 135, segmentation unit 145, and image restorationunit 155 may also be purpose built hardware such as FPGA, ASIC, etc.

FIG. 3 is a flow diagram illustrating operations performed by an imageprocessing unit 30 for detection using weighted graph cuts according toan embodiment of the present invention illustrated in FIG. 2. Imageoperations unit 115 receives a raw or a preprocessed breast image fromimage input unit 20, and performs preprocessing operations on the breastimage (S201). Preprocessing operations may include resizing,smoothening, compression, color correction, etc. Cropping unit 125receives the preprocessed breast image and crops a region of interestfrom the preprocessed breast image (S205). The cropped region ofinterest is then sent to graph generation unit 135. The cropped regionof interest may instead be sent back to image operations unit 115 formore preprocessing operations, after which it is sent to graphgeneration unit 135. Graph generation unit 135 builds a graph frompixels in the cropped region of interest (S209). Segmentation unit 145receives the cropped region of interest with the graph from graphgeneration unit 135, and performs segmentation of the pectoral musclearea in the cropped region of interest, using the associated graph(S213). Segmentation of the pectoral muscle area in the cropped regionof interest places pixels from the cropped region of interest into sets.Some of the sets are associated with the pectoral muscle, while othersets are associated with non-muscle areas, such as non-muscle breastareas, background areas, etc.

Segmentation unit 145 then performs a test to determine whether arefinement of the pectoral segmentation is to be performed, based on thecurrent segmentation results (S215). If a refinement of the pectoralsegmentation is to be performed, the cropped region of interest is sentback to cropping unit 125 or to image operations unit 115. Cropping unit125 may then crop a different region of interest from the breast image,image operations unit 115 may perform additional or new image processingoperations on the breast image or on the region of interest, graphgeneration unit 135 may build a graph from pixels in the region ofinterest again. Segmentation unit 145 then performs segmentation of thepectoral muscle area again.

Image restoration unit 155 receives the image results of pectoral musclesegmentation from segmentation unit 145, restores the received image tothe full resolution and size of the initial breast image, and outputs abreast image together with identified pectoral muscle areas/borders(S217). The output of image restoration unit 155 may be sent to imageoutput unit 50, printing unit 40, and/or display 60.

FIG. 4 is a flow diagram illustrating exemplary image processingoperations performed by an image operations unit 115 and a cropping unit125 included in an image processing unit 30 for detection using weightedgraph cuts according to an embodiment of the present inventionillustrated in FIG. 2. The flow diagram in FIG. 4 illustrates exemplarydetails of steps S201 and S205 from FIG. 3.

Image operations unit 115 receives a raw or a preprocessed breast imagefrom image input unit 20 (S301) and resizes the breast image (S303) inorder to speed up subsequent processing of the breast image. In anexemplary implementation, mammogram images at 100 um resolution receivedby image operations unit 115 are resized to 800 um. The resized breastimage is then smoothened by image operations unit 115 to reduce noise(S305), diffusion filtered to avoid loss of high spatial frequencyinformation (S307), and histogram equalized (S308). Image operationsunit 115 then outputs a processed breast image (S309). Cropping unit 125receives the processed breast image from image operations unit 115, andcrops a section of the processed breast image where the pectoral muscleis likely to be located (S311). For mammograms showing MLL/MLR views ofthe breast, the lower left or lower right portion/quadrant of the breastimage is cropped out, as the pectoral muscle is located in one of theseareas. In an exemplary set of mammograms, the cropped lower left orlower right area spans about ½ height and ⅓ width of the totalmammogram. For other views of mammograms, different sections of thebreast image may be cropped. The cropped area is the region of interest(ROI) output by cropping unit 125. The ROI is then sent to graphgeneration unit 135 (S319).

Alternative sequences for processing of the breast image may also beperformed. For example, steps S303, S305, S307 and S308 may be skippedfor the full breast image, with resizing, smoothening, diffusionfiltering, and histogram equalization performed for the ROI image, insteps S313 (resizing of ROI), S315 (smoothening of ROI), S317 (diffusionfiltering of ROI), and S318 (histogram equalization of the ROI). Inanother sequence, resizing can be performed for the full breast image(S303), while smoothening, diffusion filtering, and histogramequalization are performed for the ROI image (S315, S317, S318). Inanother sequence, smoothening, diffusion filtering, and histogramequalization can be performed for the full breast image (S305, S307,S308), while resizing is performed for the ROI image (S313). In yetanother sequence, resizing and/or smoothening, diffusion filtering, andhistogram equalization may be performed for both the full breast imageand the ROI image. In an exemplary implementation, the original breastimage, or the ROI image is smoothened with a 3×3 Gaussian window toreduce shot noise (steps S305 or S315). At the end of the sequencesshown in FIG. 4, the ROI is sent to graph generation unit 135 (S319).

In an exemplary implementation, the ROI resolution is reduced to 1/16thof its original resolution, in step S313.

FIG. 5A illustrates an exemplary mammogram image. The breast image I330in FIG. 5A illustrates a breast and a pectoral muscle section visible inthe breast image. In the breast image I330, A333 is a breast area, A335is an area of the pectoral muscle, and A331 is the background.

FIG. 5B illustrates exemplary mammogram images obtained after imageprocessing operations and cropping according to an embodiment of thepresent invention illustrated in FIG. 4. The mammogram images shown inFIG. 5B are outputs of image operations unit 115 and cropping unit 125for the input breast image I330 of FIG. 5A.

Breast image I338 is output from image operations unit 115 afterhistogram equalization of original image I330. In the histogramequalized breast image I338, A_hist333 is the breast area afterhistogram equalization, A_hist335 is the pectoral muscle area afterhistogram equalization, and A_hist331 is the background. Cropping unit125 next crops a region of interest ROI_339 from histogram equalizedimage I338. The region of interest ROI_339 is chosen to include thepectoral muscle area A_hist335. ROI_339 is magnified at the bottom ofFIG. 5B, to show the area of the pectoral muscle A_hist335.

FIG. 6 is a flow diagram illustrating operations performed by a graphgeneration unit 135 included in an image processing unit 30 fordetection using weighted graph cuts according to an embodiment of thepresent invention illustrated in FIG. 2. The flow diagram in FIG. 6describes details of step S209 in FIG. 3. Graph generation unit 135receives a cropped region of interest (ROI) of a breast image fromcropping unit 125 or from image operations unit 115, as described inFIG. 4. The ROI includes an area of the breast image with a section ofthe pectoral muscle of the breast. Graph generation unit 135 generatesrepresentations for the pixels in the ROI (S350). The pixelrepresentations may be 2-dimensional representations, orhigher-dimensional representations. Graph generation unit 135 nextselects a source point and a sink point in the ROI (S352). Graphgeneration unit 135 assigns strengths and weights to bonds betweenpixels in the ROI, including bonds between pixels, sink and sourcepoints in the ROI, using the pixel representations, the source pointlocation, and the sink point location (S354). The graph including thepixels in the ROI, the strengths and weights assigned to the bondsbetween the ROI pixels, and the sink and source points, is then sent tosegmentation unit 145 (S356).

FIG. 7A is a flow diagram illustrating exemplary operations forrepresentation of pixels for a region of interest by a graph generationunit 135 included in an image processing unit 30 for detection usingweighted graph cuts according to an embodiment of the present inventionillustrated in FIG. 6. The flow diagram in FIG. 7A illustrates exemplarydetails of step S350 from FIG. 6. Pixels representations generated instep S350 from FIG. 6 may be 2-dimensional, or higher-dimensional.

In an exemplary implementation, pixels in the ROI are optionallyrepresented in a higher-dimensional space than the 2D representation inimage space (S380). In this higher-dimensional space, pixels'representations encode more properties of pixels, besides the intensityof pixels. A pixel can, for example, be represented in a 4-dimensionalspace by Euclidean spatial coordinates, intensity, and distance from areference point (S379, S377, S375, S371). The reference point can be thelower right corner or the lower left corner of the ROI, depending on theview of the mammogram image from which the ROI was extracted (S373). Thereference point may be chosen as a pixel likely to belong to thepectoral muscle. 4-dimensional pixel representations are obtained forall pixels in the ROI.

The Euclidean spatial coordinates may be x and y coordinates, polarcoordinates, cylindrical coordinates, etc. Other higher or lowerdimensional representations of pixels, which encode moreproperties/parameters or fewer properties/parameters of pixels than thepixel properties/parameters mentioned above, may also be obtained.

FIG. 7B is a flow diagram illustrating exemplary operations forselection of a source point and a sink point in a region of interest bya graph generation unit 135 included in an image processing unit 30 fordetection using weighted graph cuts according to an embodiment of thepresent invention illustrated in FIG. 6. The flow diagram in FIG. 7Billustrates exemplary details of step S352 from FIG. 6.

To isolate the pectoral muscle from the background in a breast image,image processing unit 30 can cut the image along the pectoral muscleboundary. While this may be achieved by following local gradients, theresulting pectoral muscle segmentation may be poor. To obtain a bettersegmentation of the pectoral muscle, a global perspective that followsthe physical contours of the pectoral muscle can be used. Therepresentations of pixels obtained at step S350 in FIG. 6 are used inthe determination of contours of the pectoral muscle, because isolatingthe pectoral muscle can be reduced to isolating the pixels in thepectoral muscle in the pixel representation space. The pixelrepresentation space may be, for example, a 2-dimensional space, a4-dimensional space as described at FIG. 7A, etc.

The analogy of water flowing from a mountaintop to a lake in a valleycan be used for segmentation of objects. Flowing water follows the pathof least resistance, where a maximum flow can be achieved. Similarly,segmentation of objects can be driven from a source (an arbitrary pixelin the background) to a sink (an arbitrary spot in the pectoral muscle).The source and sink can be interchanged without consequences for theresulting segmentation.

Graph generation unit 135 selects a source pixel and a sink pixel in theROI (S391 and S397). The source pixel can be automatically selected as apixel that is most likely a background pixel and not a pectoral musclepixel (S393). The sink pixel can be automatically selected as a pixelthat is most likely a pectoral muscle pixel (S399). For example, inbreast images shown in FIGS. 5A and 5B, or breast images showing similarviews to the views in FIGS. 5A-5B, the source point can be selected asthe upper left-hand corner of the ROI image (S395), and the sink pointcan be selected as the lower right-hand corner of the ROI image (S398).

FIG. 7C is a flow diagram illustrating exemplary operations for creatinga graph with weighted edges for a region of interest by a graphgeneration unit 135 included in an image processing unit 30 fordetection using weighted graph cuts according to an embodiment of thepresent invention illustrated in FIG. 6. The flow diagram in FIG. 7Cillustrates exemplary details of step S354 from FIG. 6.

Graph generation unit 135 initializes a graph for the region of interest(S402). The graph is first populated (S404) with the source and sinkpixels obtained in step S352 of FIG. 6, or, in an exemplaryimplementation, the source and sink points obtained as shown in FIG. 7B.The graph is then populated with the pixel representations for thepixels in the ROI (S406). Graph generation unit 135 uses the pixelrepresentations generated in step S350 of FIG. 6 for the ROI pixelstogether with the source and sink pixels. In an exemplaryimplementation, graph generation unit 135 uses the higher-dimensionalpixel representations derived as described in FIG. 7A, for the ROIpixels together with the source and sink pixels. At this point, thegraph for the ROI contains points, or nodes (the pixel representations)but no edges (S410).

Graph generation unit 135 next bonds each ROI pixel to the source andsink pixels (S412). Strengths are assigned to the bonds between ROIpixels and the source and sink pixels (S414). The strength of a bond(link or edge) between an ROI pixel and the source pixel is determinedbased on the distance from the ROI pixel to the source pixel (S414). Thestrength of a bond (link or edge) between an ROI pixel and the sinkpixel is determined based on the distance from the ROI pixel to the sinkpixel (S414).

Segmentation of the pectoral muscle area in the ROI, which is describedin FIG. 8, will place pixels from the ROI into sets. Some of the setswill be associated with the pectoral muscle, while other sets will beassociated with non-muscle areas, such as non-muscle breast areas,background areas, etc. The strength of bonds based on distances of ROIpixels to source and sink pixels describe the fact that the closer a ROIpixel is to the source pixel, the more likely it is that the ROI pixelis in the same set as the source pixel. Similarly, the closer a ROIpixel is to the sink pixel, the more likely it is that the ROI pixel isin the same set as the sink pixel.

Graph generation unit 135 then creates bonds between ROI pixels (S418).Inter-pixel bonds (strength of bonds between ROI pixels) are determinedusing values of inter-pixel gradients. Additionally or alternatively,spatial distances and gradients may be used together, to form a measureof inter-pixel distance in a higher dimensional space.

Gradients between pixels are determined by generating a gradient imagefrom the ROI image (S424). The gradient image contains gradient valuesbetween pixels, and is generated using pixel intensity information inthe original mammogram. The gradient image is then used together withthe ROI image to weight the strength of the bonds between pixels. Agraph with weighted edges (bonds) between ROI pixels, source pixel, andsink pixel is obtained (S422).

Next, graph generation unit 135 assigns prior pixel label likelihood topixels in ROI (S426). Prior pixel likelihoods are obtained by amathematical relationship between the pixel location and the source (orsink). A pixel is more likely to be a source (or sink) if it isphysically closer to the source (sink). If ‘L’ represents the priorlikelihood of the pixel being of the same class as source, that is, ofthe class of background pixels, then ‘1/L’ is the likelihood that thepixel is of the same class as the sink. To calculate L (prior likelihoodfor the pixel belonging to class ‘source’), let ds be the distancebetween pixel and source, and dx be the distance between source andsink. Then L=(ds/dx). Since dx>0 and ds>0, L>0 and 1/L<infinity.

The graph generation unit 135 then outputs a fully connected graph withweighted edges to segmentation unit 145 (S428).

FIG. 8 is a flow diagram illustrating operations performed by asegmentation unit 145 included in an image processing unit 30 fordetection using weighted graph cuts according to an embodiment of thepresent invention illustrated in FIG. 2. A min-cut max-flow method maybe used to segment the ROI by finding the cheapest cut (max flow) fromthe source node to the sink node and obtain an area of pectoral musclein the ROI.

The min-cut max-flow algorithm uses the inter-pixel bond weights, thepixel-source, and pixel-sink bond weights to arrive at a segmentation ofthe ROI (S506) with a global maximum flow. The algorithm cuts edges(breaks bonds) between pixels belonging to different sets (classes). Theoutput of segmentation unit 145 after performing the min-cut max-flowalgorithm is a ROI in which pixels have been assigned to two sets(classes), with a set (class) of pixels representing the pectoralmuscle, while another pixel set (class) is associated with otherfeatures of the ROI, such as the breast and the background.

Segmentation unit 145 may perform min-cut max-flow segmentation of theROI using methods described in “An Experimental Comparison ofMin-Cut/Max-Flow Algorithms for Energy Minimization in Vision”, by Y.Boykov and V. Kolmogorov, IEEE Transactions on PAMI, v. 26 No. 9, pp1124-1137, September 2004, the entire contents of which are herebyincorporated by reference. As discussed in the above publication, a cutC on a graph with two terminals is a partitioning of the nodes in thegraph into two disjoint subsets S and T such that the source s is in Sand the sink t is in T. Such a cut C is also knows as an s/t cut. Thecost of a cut C={S,T} is defined as the sum of the costs of “boundary”edges (p,q) where pεS and qεT. The minimum cut problem on a graph is tofind a cut that has the minimum cost among the possible cuts. Incombinatorial optimization it has been shown that the minimum s/t cutproblem can be solved by finding a maximum flow from the source s to thesink t. Using an analogy, a maximum flow is a maximum “amount of water”that can be sent from a source to a sink by interpreting graph edges asdirected “pipes” with capacities equal to edge weights.

In the case of ROI from breast images showing the pectoral muscle, whenthe sink t is a pectoral muscle pixel, the subset T is the set (class)of pixels that belong to the pectoral muscle (S508). In that case, thesource s is a background pixel, and the subset S is the set (class) ofpixels that do not belong to the pectoral muscle, so subset S caninclude pixels from the breast, the background, etc.

Segmentation unit 145 next converts the segmented graph obtained (thegraph together with the segmentation results) back to a mask image(S512), and outputs a binary image showing the pectoral muscle (S516).The binary image is sent to image restoration unit 155.

In alternative embodiments, segmentation unit 145 may segment the ROIusing other energy minimizing functions besides, or instead of, amin-cut/max-flow segmentation.

FIG. 9 is a flow diagram illustrating operations performed by an imagerestoration unit 155 included in an image processing unit 30 fordetection using weighted graph cuts according to an embodiment of thepresent invention illustrated in FIG. 2. Image restoration unit 155receives from segmentation unit 145 a binary image showing the pectoralmuscle. The binary image corresponds to the ROI that was extracted fromthe initial breast image. Image restoration unit 155 restores the binaryimage including the pectoral muscle to the full resolution of theuncropped initial breast image, and outputs a pectoral muscle mask inthe original space of the original breast image. The results of imagerestoration unit 155 are sent to printing unit 40, image output unit 50,and/or display 60.

Although aspects of the present invention have been described in thecontext of mammography images, it should be realized that the principlesof the present invention are applicable to other types of digital imagesbesides mammography images, for detection of various other types ofobjects besides pectoral muscles.

Although detailed embodiments and implementations of the presentinvention have been described above, it should be apparent that variousmodifications are possible without departing from the spirit and scopeof the present invention.

1. An image processing method, said method comprising: accessing digitalimage data representing an image including an object; generating aconnected graph associated with said image, said generating stepincluding representing pixels of said image in a higher than twodimensional space to obtain pixel representations, generating a pixelrepresentation graph using said pixel representations, and assigningweights to edges between said pixel representations in said pixelrepresentation graph, based on a gradient characteristic between saidpixel representations, to obtain a connected graph; and segmenting saidconnected graph using an energy minimizing function, to obtain pixels ofsaid image associated with said object.
 2. The image processing methodas recited in claim 1, wherein said segmenting step uses a max-flowsegmentation to obtain pixels of said image associated with said object.3. The image processing method as recited in claim 1, wherein saidsub-step of generating a pixel representation graph selects a sourcepixel from among pixels of said image not included in said object, and asink pixel from among pixels of said image included in said object. 4.The image processing method as recited in claim 3, wherein said step ofgenerating a connected graph further comprises: assigning strengths,based on a distance characteristic, to edges between said pixelrepresentations in said pixel representation graph, a sink pixelrepresentation associated with said sink pixel, and a source pixelrepresentation associated with said source pixel.
 5. The imageprocessing method as recited in claim 4, wherein said image is a digitalmammogram, said object is a pectoral muscle in said digital mammogram,said source pixel is a background pixel, said sink pixel is a pixel ofsaid pectoral muscle.
 6. The image processing method as recited in claim1, wherein said step of generating a connected graph further comprises:assigning strengths to edges between said pixel representations, basedon a distance characteristic between said pixel representations.
 7. Theimage processing method as recited in claim 1, wherein said step ofgenerating a connected graph further comprises: assigning prior pixellabel likelihoods to said pixel representations in said connected graph.8. The image processing method as recited in claim 1, furthercomprising: preprocessing said image by resizing, smoothening, anddiffusion filtering said image, before said step of generating aconnected graph.
 9. The image processing method as recited in claim 8,further comprising: cropping from said image a region of interestassociated with said object, before said step of generating a connectedgraph.
 10. The image processing method as recited in claim 9, whereinsaid step of generating a connected graph and said segmenting step areperformed for said region of interest to obtain a segmented region ofinterest.
 11. The image processing method as recited in claim 10,further comprising: integrating said segmented region of interest backinto said image after said segmenting step, and restoring said image tooriginal resolution.
 12. The image processing method as recited in claim1, wherein said sub-step of representing pixels represents pixels ofsaid image using a parameter relating to a spatial characteristic ofsaid pixels in said image, a parameter relating to an intensitycharacteristic of said pixels in said image, and a parameter relating toa distance characteristic of said pixels to a reference point.
 13. Theimage processing method as recited in claim 1, wherein said image is adigital mammogram and said object is a pectoral muscle in said digitalmammogram.
 14. An image processing method, said method comprising:accessing digital image data representing an image including an object;generating a connected graph associated with said image, said generatingstep including representing pixels of said image to obtain pixelrepresentations, generating a pixel representation graph using saidpixel representations, and assigning weights to edges between said pixelrepresentations in said pixel representation graph, based on a gradientcharacteristic between said pixel representations, to obtain a connectedgraph; and segmenting said connected graph using an energy minimizingfunction, to obtain pixels of said image associated with said object.15. The image processing method as recited in claim 14, wherein saidsub-step of generating a pixel representation graph selects a sourcepixel from among pixels of said image not included in said object, and asink pixel from among pixels of said image included in said object, andsaid segmenting step uses a max-flow segmentation to obtain pixels ofsaid image associated with said object.
 16. The image processing methodas recited in claim 15, wherein said step of generating a connectedgraph further comprises: assigning strengths to edges between said pixelrepresentations in said pixel representation graph, a sink pixelrepresentation associated with said sink pixel, and a source pixelrepresentation associated with said source pixel, based on a distancecharacteristic.
 17. An image processing apparatus, said apparatuscomprising: an image data input unit for accessing digital image datarepresenting an image including an object; a graph generator forgenerating a connected graph associated with said image, said graphgenerator generating a connected graph by representing pixels of saidimage to obtain pixel representations, generating a pixel representationgraph using said pixel representations, and assigning weights to edgesbetween said pixel representations in said pixel representation graph,based on a gradient characteristic between said pixel representations,to obtain a connected graph; and a segmentation unit for segmenting saidconnected graph using a max-flow segmentation to obtain pixels of saidimage associated with said object.
 18. The apparatus according to claim17, wherein said graph generator selects a source pixel from amongpixels of said image not included in said object, and a sink pixel fromamong pixels of said image included in said object for said pixelrepresentation graph.
 19. The apparatus according to claim 18, whereinsaid graph generator assigns strengths, based on a distancecharacteristic, to edges between said pixel representations in saidpixel representation graph, a sink pixel representation associated withsaid sink pixel, and a source pixel representation associated with saidsource pixel.
 20. The apparatus according to claim 19, wherein saidimage is a digital mammogram, said object is a pectoral muscle in saiddigital mammogram, said source pixel is a background pixel, said sinkpixel is a pixel of said pectoral muscle.
 21. The apparatus according toclaim 17, wherein said graph generator assigns strengths to edgesbetween said pixel representations, based on a distance characteristicbetween said pixel representations.
 22. The apparatus according to claim17, wherein said graph generator assigns prior pixel label likelihoodsto said pixel representations in said connected graph.
 23. The apparatusaccording to claim 17, further comprising: an image operations unit forpreprocessing said image by resizing, smoothening, and diffusionfiltering said image, before said graph generator generates saidconnected graph.
 24. The apparatus according to claim 23, furthercomprising: a cropping unit for cropping from said image a region ofinterest associated with said object, before said graph generatorgenerates said connected graph.
 25. The apparatus according to claim 24,wherein said graph generator generates a connected graph for said regionof interest, and said segmentation unit segments said connected graphfor said region of interest to obtain a segmented region of interest.26. The apparatus according to claim 25, further comprising: an imagerestoration unit for integrating said segmented region of interest backinto said image after said segmenting step, and restoring said image tooriginal resolution.
 27. The apparatus according to claim 17, whereinsaid graph generator represents pixels of said image in a higher thantwo dimensional space using a parameter relating to a spatialcharacteristic of said pixels in said image, a parameter relating to anintensity characteristic of said pixels in said image, and a parameterrelating to a distance characteristic of said pixels to a referencepoint.
 28. The apparatus according to claim 17, wherein said image is adigital mammogram and said object is a pectoral muscle in said digitalmammogram.